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>ensation and waveform coding field by field. 
^KOFP^ m recent years, for reproducing an image signal for 
each of objects in a frame image while increasing efficiency 
in compressing an image signal, a coding method has beerg 
conceived, wherein image signals of the objects are | 
compressives coded and transmitted separately for each*| ^ 
object. A coded image signal which has been coded by thi| g g 
coding method is decoded at the reproducing end in a manneS 
adapted to the coding method. Specifically, in this 
decoding, coded image signals of respective objects are 
decoded, and the resulting decoded image signals of 
respective objects are synthesized, to produce a reproduced 
image signal. Then, based on the reproduced image signal, 
a frame image comprising respective objects is displayed. 

By using the coding method for performing coding to the 
image signal for each object as described above, it becomes 
possible to combine objects freely to generate a composite 
image,, whereby moving images can be reedited with ease. In 
addition, a moving, image comprising only objects with high 
importance can be displayed without reproducing objects with 
relatively low importance, depending on&egree of congestion 
of a communication path, performance of a reproducing 
apparatus, or preference of a viewer. 

For coding an image signal of an object, (an image having 
an arbitrary shape) /waveform transform which performs 
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signal processing adapting to a shape of the object, for 
example, shape adaptive DCT, or waveform transform to a 
padded image signal, is employed. In the method, 
specifically, padding is performed to an image signal forming 
an image space (rectangular region) by replacing values of 
pixels in its insignificant region with padding pixel values 
obtained by a prescribed method, and then^onventional 8 x 
8 cosine transform is performed to .the padded image signal. 
The insignificant region is a region outside an object in 
the rectangular region, and comprises pixels which have no 
values for displaying the object. In other words, an image 
signal of the insignificant region comprises so-called 
insignificant sample values. Also, the 8 x 8 cosine 
transform is^aveform transform which performs cosine 
transform to the image signal of the rectangular region for 
each image space comprising 8 x 8 pixels. ^ 

As" a method for eliminating redundant signal^ between 
adjacent frames, there is a method for obtaining'dif f erence 
between an image signal of a target macroblock to be coded 
and its prediction signal for each image space (macroblock) 
comprising 16 x 16 pixels, as a prediction error signal 
(difference signal). Here, it is assumed that the 
prediction signal is an image signal ot ^ prediction region 
obtained by motion compensation. The motion compensation 
process detects a region comprising 16 x 16 pixels 
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corresponding to an image signal in a coded or decoded frame 
where^dif ference between the image signal of the target 
macroblock and the image signal is the smallest, as a 
prediction region. 

In some cases, however, the prediction region also 
includes pixels (hereinafter referred to as insignificant 
pixels) which have insignificant sample values. In such 
cases, when the difference between the prediction signal of 
the prediction region including insignificant values and an 
image signal of the target region is computed, difference 
values tend to be very large, since the sample values of the 
insignificant pixels are not always optimal prediction 
values for smaller difference. 

As a solution to this, padding is performed to the. image 
signal of the prediction region by replacing insignificant 
sample values by prescribed padding values, and then 
^difference between the padded prediction signal and the image 
signal of the target macroblock is computed as a difference 
signal (prediction error signal), which is subjected to 
transformation for coding. This Padding of the prediction 
signal can suppress the difference signal. 

In the prior art pixel value padding, as a padding value 
for an insignificant sample value, sample values of 
significant values (pixels having significant sample values) 
adjacent to the corresponding insignificant pixel in the 
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vertical and horizontal directions, respectively, are 
averaged, and the resulting average is used. By the use of 
the average of the sample values of the significant pixels, 
high-frequency components into an image signal (prediction 
signal) in a padded image space can be suppressed, thereby 
increasing coding efficiency. 

However, if padding which uses the average of the 
significant pixels adjacent to the insignificant pixels in 
the vertical and horizontal directions as the padding value 
is applied to the image signal of the interlaced image, 
high-frequency components of the image signal increase, 
which will be described below in detail. 

In the interlaced image, especially in a case where 
motion of an object is large, there is low correlation 
of pixel values between adjacent scanning lines. Figure 18 
schematically shows an example of "arrangement of pixel values 
which shows t>crr low correlation of pixel values between 
adjacent scanning lines in the interlaced image, as 
arrangement of pixel values in an image space 301. 

The image space 301 is an image space comprising 8 x 
8 pixels. In the figure, each pixel is represented as a 
square. Pixels (for example pixels 303 and 304) of hatched 
squares are pixels (significant pixels) having significant 
sample values, and pixels (for example pixel 302) of squares 
which are not hatched are pixels (insignificant pixels) 
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having insignificant pixels. The number of each square 
indicates a sample value of the corresponding pixel. 

Since scanning timings differ from each other between 
odd-numbered and even-numbered fields in the interlaced 
image, in a case where motion of an object in an image is 
large, it looks like the object has a shape of two outlines 
in a frame composed of these two fields, as it can be observed 
in the image space 301 in figure 18. In this case, for 
example, the significant pixel 304 is adjacent to its upper 
and lower insignificant pixels. /?$ £/<^f'' c ) 

Next, a prior art padding method. and its ^Ul-ing result 
will be described with reference to figures 18 and 19. 

In the prior art padding method, three steps are 
performed . 

In a first step SI, sample values of insignificant 
pixels are replaced with sample values of significant pixels 
adjacent thereto in the image space 301, for insignificant 
pixels aligned in the horizontal direction, in the order from 
inward to outward of the image space 301. Then, sample 
values of insignificant pixels in the image space to which 
this horizontal replacement process has been performed are 
replaced with sample values of significant pixels adjacent 
thereto, for insignificant pixels aligned in the vertical 
direction, in the order from inward to outward of the image 
space. Process in the first step SI thus performed to the 
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in figure 18. 

More specifically, in the vertical replacement process 
in the second step S2, sample values of insignificant pixels 
on pixel columns 362-365 are replaced with padding values 
in the image spacS 351. In addition, in the horizontal 
replacement process in the second step S2, sample values of 
insignificant pixels on pixel columns 360 and 361 are 
replaced with sample values of significant pixels and padded 
pixels on the pixel column 362, and likewise, sample values 
of insignificant pixels on pixel columns 367 and 368 are 
replaced with sample values of significant pixels and padded 
pixels on the pixel column 365. 

in a third step S3, as shown in figure 19, sample values 
of pixels in the image space 350 obtained through the first 
step SI and the corresponding pixels in the image space 351 
obtained through the second step S2 are averaged, which 
results in an image space 380 comprising pixels having sample 

values in figure 19. 

Resampling which divides pixels forming the image space 
380 into pixels corresponding to odd-numbered and 
even-numbered fieldsmen assumption that the image space 380 
is a frame, results in an image space 381 comprising plural 
pixels corresponding to the odd-numbered field and an image 
space 382 comprising plural pixels corresponding to the 
even-numbered field as shown in figure 20. In the image 
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spaces 381 and 382 corresponding to these respective fields, 
sample values of pixels are non-uniform, which introduces 
high-frequency components into image signals in image spaces 
381 and 382. 

in addition, this problem associated with the padding 
occurs in the non-interlacing scanned image (progressive 
image) as well as in the interlacing scanned image 
(interlaced image). Specifically, in many cases, the 
progressive image has a stripe pattern, and for the case of 
the progressive image having the stripe pattern, the above 
resampling is performed in such a manner that plural pixels 
forming a frame (image space) are collected into a stripe 
portion and the other portion in the image space, to form 
image spaces corresponding to the stripe pattern portion and 
the other portion, and then the resulting resampled image 

spaces are coded. 

inthis case, if padding is performed to the image signal 
of the progressive image having the stripe pattern by the 
prior art padding method, high-frequency components are 
introduced into an image signal corresponding to the 
resampled image space, leading to degraded coding 
efficiency. 

Further, £e similar problem occurs in padding methods 
other than the above, which . computes padding values using " 
significant sample values adjacent to insignificant pixels 
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the horizontal or vertical direction. 

The present invention is directed to solving the 
above-mentioned problem and has an object to provide a 
digital image padding method in which padding is performed 
to an interlacing scanned image or an image of a stripe pattern 
having an arbitrary shape while suppressing degradation of 
efficiency in coding the prediction error signal, an image 
processing apparatus which performs image coding or image 
decoding by the padding method, and a data recording medium 
which contains a program for implementing these image 
processing by the padding method or the image processing 
apparatus in a computer. 

Di^eWof thejnvent^ . 

wording to ^ f P£g# * 
image padding method ^^^-^±n^±on for performing 
padding-of pixel values to a digital image signal forming 
an original image space comprising an image having an 
arbitrary shape and comprising significant pixels and 
insignificant pixels, comprises a pixel rearrangement step 
in which plural pixels in the original image space are grouped 
according to a prescribed rule, to for, plural small image 
spaces each comprising pixels of the same group; and a pixel 
padding step in which values of insignificant pixels in each 
small image space are replaced with padding pixel values 
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generated by a prescribed method. 

In the digital image padding method thus constructed, 
plural pixels forming the original image space are extracted 
according to a prescribed rule, to form plural small images. 
In other words, it is possible to divide the original image 
space where there is low correlation of pixel values between 
adjacent pixels or scanning lines into plural small images 
where there is high correlation of pixel values between the 
adjacent pixels or scanning lines. Accordingly, padding is 
performed to the interlacing scanned image or the image of 
a stripe pattern having an arbitrary shape for each small 
image space with high correlation of the pixel values rather 
than the original image space with low correlation of pixel 
values between adjacent pixels or scanning lines. In this 
case, in the padded small image, there is high correlation 
of the pixel values, and it is possible to avoid introduction 
of high-frequency components into the image signal 
corresponding to the original image space in process for 
padding the interlacing scanned image or the image of a stripe 
pattern having an arbitrary shape. 

As a result, differential coding or differential 
decoding is performed to the interlacing scanned image while 
suppressing degradation of coding efficiency resulting from 

the padding. ^ <***ecf 

According to^e d - aim 2 of the present invention, in the 
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digital image padding method of oieim— 1 , in the pixel 
rearrangement step, sampling is performed by extracting 
pixels every (N + l)-th (N: positive integer) pixel in a 
prescribed direction of the image space (N + 1) times by using 
the first to the (N + l)-th pixels as starting sample pixels, 
to form (N + 1) pieces of small image spaces each comprising 
a prescribed number of pixels obtained by one sampling. 

In the digital image padding method thus constructed, 
pixels in the original image space are extracted at intervals 
of a prescribed number of pixels in a prescribed direction 
of the original image space, to form the small image spaces. 
Therefore, it is possible to divide the original image space 
comprising an image having an arbitrary shape of a vertical 
stripe pattern or a horizontal stripe pattern into small 
image spaces where there is high correlation of pixel values 
between adjacent pixel rows or between adjacent pixel columns, 
irrespective of a width or arrangement spacing of stripes. 

According to cl ai m s of the present invention, in the 
digital image padding method of operation values y^^^, 

obtained by performing operation on values of significant 
pixels in an N-th (n=l, 2,..., N + l) small image space are used 
as the padding pixel values for replacing the values of 
insignificant pixels in the n-th small image space. 

In the digital image padding method thus constructed, 
values of insignificant pixels in a small image space are 
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replaced with padding pixel values obtained from values of 
significant pixels in the same small image. Therefore, it 
is possible to suppress generation of pixel values which are 
significantly different from values of significant pixels 
in the small image space in padding process. 

According to of the present invention the 

digital image padding method of c la im 2 further comprises 
an image space restoring step in which plural pixels forming 
padded small image spaces are rearranged following a rule 
adapted to the prescribed rule of the grouping so that these 
pixels forms a restored image space which has the same pixel 
rearrangement as the original image space. 

In the digital image padding method thus constructed, 
after the original image space is divided into plural small 
images where there is high correlation of pixel values, and 
padding. of pixel values is performed to these small image 
spaces, 'pixels in the padded small image space are rearranged 
so that they have the same arrangement as the original image 
space. Therefore, the image signal corresponding to the 
padded original image space can be processed in the same 
manner that the image signal corresponding to the unpadded 
original image space is, Processed. 

According to c^m-S-^f the p. res e^inven txon,^n the ^ 
digital image padding method of ^rn^4 , operation values 
obtained by performing operation on values of significant 
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pixels in an n-th (n=l, 2,..., N + l) small image space are used 
as the padding pixel values for replacing the values of 
insignificant pixels in the n-th small image space. 

In the digital image padding method thus constructed, 
values of insignificant pixels in a small image space are 
replaced with padding pixel values obtained from values of 
significant pixels in the same small image space. Therefore, 
it is possible to suppress generation of pixel values which 
are significantly different from values of significant 
pixels in the small image space in padding process. 

According to «ir^rm-6 of the present invent ion , . in the 
digital image padding method of elaim 1, in the pixel /Hue*- 
rearrangement step, sampling is continuously performed twice 
using the first and second pixel rows as starting sample pixel 
rows, by extracting pixels on every other pixel row in the 
vertical direction of the original image space, to form first 
and second small image spaces each comprising a prescribed 
number of pixels obtained by the first and second samplings, 
respectively . 

In the digital image padding method thus constructed, 
every other pixels are extracted in the vertical direction 
of the original image space, to form a small image space 
composed of odd-numbered scanning lines in the original image 
space and a small image space composed of even-numbered 
scanning lines and a small image space composed of even- 
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numbered scanning lines. Therefore, padding of the 
interlacing scanned image is performed for each small image 
space corresponding to each field where there is high 
correlation of pixel values between adjacent scanning lines 
rather than the original image space where there is low 
correlation of pixel values between them. As a result, it 
is possible to avoid introduction of high-frequency 
components into an image signal of the interlacing scanned 
image in padding process. /i^^^c 

According to c-jr aim 7 of the present invention, a digital 
image padding method for performing padding of pixel values 
to a digital image signal forming an original image space 
comprising an image having an arbitrary shape and comprising 
significant pixels and insignificant pixels, comprises the 
steps of dividing the original image space into a first small 
image space comprising pixels on odd-numbered pixel rows in 
the original image space and a second small image space 
comprising pixels on even-numbered pixel rows in the original 
image space; and generating first padding pixel values from 
values of significant pixels in the first small image space 
and replacing values of insignificant pixels in the first 
small image space with the first padding pixel values, and 
generating second padding pixel values from values of 
significant pixels in the second small image space and 
replacing values of insignificant pixels in the second small 
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image space with the second padding pixel values. 

In the digital image padding method thus constructed, 

padding of the interlacing scanned image is performed for 

each of first and second small image spaces corresponding 

to fields where there is high correlation of pixel values 

between adjacent scanning lines rather than the original 

image space where there is low correlation of pixel values 

between them. As a result of this, it is possible to avoid 

introduction of components into an image signal of the 

interlacing scanned image in padding process. 

Besides, values of insignificant pixels in each small 

image space are replaced with padding pixel values obtained 

from values of significant pixels in the same small image 

space, and it is therefore possible to suppress generation 

of pixels values which are significantly different from 

values of significant pixels in the small image space in 
i 

padding ' process . , .,x->^~^- 

According to &laim -8-of the present invention, a digital 
image padding method for performing padding to a digital 
image signal forming an original image space comprising an 
image having an arbitrary shape and comprising significant 
pixels and insignificant pixels, by replacing values of 
insignificant pixels with padding pixel values determined 
from values of significant pixels in the original image space, 
wherein, significant pixels which have values used for 
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determining the padding pixel values are pixels other than 
pixels adjacent to insignificant pixels to be padded. 

In the digital image padding method thus constructed, 
the padding pixel values for replacing values of 
insignificant pixels are determined from values of 
significant pixels other than pixels adjacent to the 
insignificant pixels in the original image space, and 
therefore the values of the insignificant pixels are not 
replaced with the values of the pixels adjacent thereto. As 
a result, it is possible to perform padding of pixel values 
to the original image space where there is low correlation 
of pixel values between adjacent pixels or scanning lines 
while suppressing introduction of high-frequency components 
into the image signal corresponding to the original image 

space - * »^H» 

According to ©4^itn-^of the present invention, in the 
digital 'image padding method of-Gia-iffi-8, significant pixels' 
which have values used for determining the padding pixel 
values are one pixel apart from the insignificant pixels to 
be padded. 

In the digital image padding method thus constructed, 
values of insignificant pixels on an odd-numbered scanning 
line are replaced with padding pixel values obtained from 
values of significant pixels on another odd-numbered 
scanning line in the original image space, and values of 
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insignificant pixels on an even-numbered scanning line are 
replaced with padding pixel values obtained from values of 
significant pixels on another even-numbered scanning line 
in the original image space. Therefore, it is possible to 
avoid introduction of high-frequency components into the 
image signal corresponding to the interlacing scanned image 
in padding process. . 

According to -ei-a^m^ of the present invention, an image 
processing apparatus for per f orming^image coding which 
comprises "correlation identif ying_jn^ns for identifying 
correlation between pixel values of a digital image signal 
and outputting a sample identifier according to the^ 
identification result, a coding section for performing 
differential coding to the digital image signal by the use 
of a prediction image signal of the digital image signal 
according to the sample identifier and outputting a coded 
image signal, and a predicting section for producing the 
prediction image signal based on a decoded image signal in 
which the coded image signal has been locally decoded, the 
coding section comprises a subtracter for computing 
difference between the digital image signal and the 
prediction image signal as a difference signal; data 
compressing m£«fs ' ffr compressing the difference image 
signal to produce a compressed image signal; and a variable 
length encoder for performing variable length coding to the 
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compressed difference signal, and the predicting section 

comprises data expanding jjieans for expanding the compressed 

difference signal to produce an expanded difference signal; 

an adder for adding the expanded difference signal to the 

prediction image signal to produce a decoded image signal; 

•'padding nj^erfis for performing padding according to the sample 

identifier by rearranging pixels in an image space formed 

by the decoded image signal and replacing values of 

insignificant pixels in the image space where the pixels have 

been rearranged with padding pixel values generated by a 

// ^ 

prescribed method; and ^predict ion image signal producing 
jus^ms for producing the prediction image signal from the 
padded decoded image signal according to the sample 
identifier . 

In the digital image processing apparatus thus 
constructed, the predicting section for producing the 
prediction image signal of the digital image signal, 
comprises the padding gusnams for performing padding of pixel 
values to the decoded image signal in which the coded 
difference signal has been locally decoded in its 
differential coding process, and is used for producing the 
prediction image signal from the padded decoded image signal, 
the padding nve^ns first rearranging pixel values of the 
decoded image signal and then replacing the values of the 
insignificant pixels of the decoded image signal with the 
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prescribed padding pixel values. Therefore, when 

performing differential coding to the digital image signal 

of the interlacing scanned image or the image of a stripe 

pattern having an arbitrary shape, it is possible in padding 

process to avoid introduction of high-frequency components 

into the prediction image signal for use by differential 

coding, differential coding can be performed to the 

interlacing scanned image while suppressing degradation of 

coding efficiency resulting from the' padding . ^ 

According to claim - 1-1 of the present invention, in the 

7*<^:. < ^^^ ^ & ^ :*^<r 

image processing apparatus of ^ie-im-iO , the coding section^^//^ 

further compri s es K padding rr\^ans for performing padding to 

the difference image signal according to the sample 

identifier by rearranging pixels in the image space formed 

by the difference image signal and replacing values of 

insignificant pixels in the image space where the pixels have 

been rearranged with padding pixel values generated by a 

prescribed method, and the data compressing ip^ans performs 

information compression to the padded difference image 

signal to produce a compressed difference image signal. 

In the image processing apparatus thus constructed, 

pixel values of the difference image signal input to the data 

compressing rjL&^rfis are rearranged, and then padding is 

performed by replacing values of the insignificant pixels 

of the difference signal with the prescribed padding pixel 
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values. Therefore, it is possible to avoid introduction of 
high-frequency components into the image signal input to the 
data compressing means. As a result, when performing 
differential coding to the digital image signal of the 
interlacing scanned image or the image of a stripe pattern 
having an arbitrary shape, waveform transform such as DCT 
is performed with high efficiency. 

According to e l aim 1 2 of the present invention, an image 
processing apparatus for performing image decoding which 
comprises a reproducing section for reproducing a coded image 
signal in which a digital image signal has been coded by the 
use of a prediction image signal of the digital image signal 
and outputting a reproduced image signal, and a predicting 
section for producing the prediction image signal from the 
reproduced image signal, and the reproducing section 
comprise a data analyzing unit for analyzing the coded image 
signal to extract a compressed difference signal of the 
digital image signal and a sample identifier indicating 
correlation between pixel values of the digital image signal 
from the coded image signal; data expanding joe-aYis for 
expanding the compressed difference signal to produce an 
expanded difference signal; and an adder for adding the 
expanded difference signal to the prediction signal to 
produce the reproduced image signal, and in the predicting 
section, using at least one of the reproduced image signal 
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and the prediction image signal obtained from the reproduced 
image signal as a signal to be padded, padding is performed 
to the signal to be padded according to the sample identifier, 
by rearranging pixels in an image space formed by the signal 
to be padded and replacing values of insignificant pixels 
in the image space where the pixels have been rearranged with 
padding pixel values generated by a prescribed method. 

In the image processing apparatus thus constructed, the 
predicting section for producing the prediction image signal 
of the digital image signal comprises "padding xn&&t\s for 
performing padding of pixel values to the reproduced image 
signal or the prediction image signal of the digital image 
signal as the signal to be padded, and the padding process 
rearranges pixel values of the signal to be padded and then 
replaces values of insignificant pixels of the signal to be 
padded with prescribed padding pixel values. Therefore, 
when decoding a coded difference signal of the interlacing 
scanned image or the image of a stripe pattern having an 
arbitrary shape, it is possible to avoid introduction of 
high-frequency components into the prediction image signal 
for use by the differential decoding. As a result, 
differential decoding adapted to differential coding which 
suppresses degradation of coding efficiency resulting from 
padding of pixel values, is realized. - 

According to o-laim-3r3 of the present invention, in the 
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image processing apparatus of Gia-hn — 3r2 , the predicting 

»/^ CtktS/ Re- 

section further comprises ^padding m^arris for performing 

padding to the reproduced image signal frame by frame or field 

by field according to the sample identifier, and a frame 

memory for storing a padded reproduced signal in which the 

reproduced image signal has been padded, and outputs the 

padded reproduced image signal stored in the frame memory 

to the reproducing section as the prediction image signal. 

In the image processing apparatus thus constructed, 
padding is performed to the reproduced image signal according 
to the sample identifier frame by frame or by field by field. 
Therefore, especially when performing differential decoding 
to the digital image signal of the interlacing scanned image, 
it is possible to avoid introduction of high-frequency 
component into the prediction image signal obtained from the 
reproduced image signal in padding process. 

According to -ejre riim 1 -4- of the present invention, in the 
image processing apparatus of c lai m — ¥2 , the predicting / s?t*£-&>>Y/ <j 
section further comprises a frame memory for storing the 
reproduced image signal, and padding jae^hs for performing 
padding to the prediction image signal extracted from the 
reproduced image signals stored in the frame memory frame 
by frame or field by field according to the sample identifier, 
and outputs the padded prediction image signal to the 
reproducing section. 
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In the image processing apparatus thus constructed, 
padding is performed to the prediction image signal according 
to the sample identifier frame by frame or by field by field. 
Therefore, especially when performing differential decoding 
to the digital image signal of the interlacing scanned image, 
it is possible to avoid introduction of high-frequency 
components into the prediction image signal in the process 
for padding the prediction image signal. Besides, since 
padding is performed only to the prediction signal fetched 
from the frame memory, amount of signals to be padded in the 
predicting section can be reduced. ^ 

According to G4rarim-i-S- o f the present invention, an image 
processing apparatus for performing image decoding, which 
comprises a reproducing section for reproducing a coded image 
signal in which a digital image signal has been coded by the 
use of a prediction image signal of the digital image signal 
and outputting a reproduced image signal, and a predicting 
section for producing the prediction image signal from the 
reproduced image signal, the reproducing section comprises 
a data analyzing unit for analyzing the coded image signal 
to extract a compressed difference signal of the digital 
image signal and a sample identifier indicating correlation 
between pixel values of the digital image signal from the 
coded image signal; data expanding n^e-srns for expanding the 
compressed difference signal to produce an expanded 
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difference signal,; and an adder for adding the expanded 
difference signal to the prediction image signal to produce 
the reproduced image signal, and the predicting section 
comprises padding rriHns for performing padding to the 
reproduced image signal according to the sample identifier, 
by replacing values of insignificant pixels in an image space 
formed by the reproduced image signal with padding pixels 
values generated from values of significant pixels other than 
pixels adjacent to the insignificant pixels; and 
rearrangement m^ns for rearranging pixels in an image space 
formed by the padded reproduced image signal, according to 
the sample identifier. 

In the digital image processing apparatus thus 
constructed, when performing padding to the reproduced image 
signal obtained from the coded difference image signal 
containing digital image information, the padding pixel 
values for replacing values of insignificant pixels are 
determined from values of significant pixels other than 
pixels adjacent to the insignificant pixels in the image 
space formed by the reproduced image signal. Therefore, 
values of insignificant pixels will not be replaced with 
values of pixels adjacent thereto. As a result, when 
performing differential decoding to the digital image signal, 
it is possible to perform padding of pixel values in the 
original image space where there is low correlation of pixel 
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values between adjacent pixels or scanning lines while 
suppressing introduction of high-frequency components into 
the image signal in the original image space. 

k According to e - laiiu 3n5 of the present invention, a data 

recording medium for storing a program which makes a computer 
perform padding of pixel values to a digital image signal 
forming an original image space comprising an image having 
an arbitrary shape and comprising significant pixels and 
insignificant pixels, and the program makes the computer 
perform padding of the pixel values according to the digital 
image padding method of claim -1 . s^**^****?***? 

I ^ By loading the program stored in the data recording 
medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image or 
the like. 

' ^ According to GJr aim 1 ?- of the present invention, a data 
recording medium for storing a program which makes a computer 
perform padding of pixel values to a digital image signal 
forming an original image space comprising an image having 
an arbitrary shape and comprising significant pixels and 
insignificant pixels, and the program makes the computer 
perform padding of pixel values according to the digital 
image padding method of e&£fl . ***~*^+> ^/^^ ^ 
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By loading the program stored in the data recording 
medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image. 

According to claim l -8~~of the present invention, a data 
recording medium for storing a program which makes a computer 
perform padding of pixel values to a digital image signal 
forming an original image space comprising an image having 
an arbitrary shape and comprising significant pixels and 
insignificant pixels, and the program makes the computer 
perform padding of pixel values according to the digital 
image padding method of claim 8 . ~ s^r^&^s ^^-j 

I By loading the program stored in the data recording 

medium thus constructed into the computer, padding is 
implemented by the computer without degrading coding 
efficiency when performing differential coding or 
differential decoding to the interlacing scanned image. 
/( v According to c - laim 1 -9- of the present invention, a data 

recording medium for storing a program which makes a computer 
perform image signal coding, and the program makes the 
computer perform differential coding to a digital image 
signal by the image processing apparatus of -eiaira — 1-0. ^"V^ .---v^Uc- 

By loading the program stored in the data recording 
medium so constructed into the computer, differential coding 
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of the interlacing scanned image or the like is implemented 
by software with no degradation of coding efficiency 
resulting from padding. 

According to e-3^a-3r m 2 0 of the present, invention, a data 
recording medium for storing a program which makes a computer 
perform image signal decoding, and the program makes the 
computer perform differential decoding to the digital image 
signal by the image processing apparatus of cl aim — 1-2. QT' *f"'0^ 

By loading the program stored in the data recording 
medium thus constructed, differential decoding of the 
interlacing scanned image, which is adapted to differential 
coding with no degradation in coding efficiency resulting 
from padding, is implemented by software. 

According to Qrjr aim 21 of the present invention, a data 
recording medium for storing a program which makes a computer 
perform image signal decoding, and the program makes the 
computer perform differential decoding to the digital image 

signal by the image processing apparatus of e- l ai m — ir5~? '>/ ^ 

fy -/ Ac 

By loading the program stored in the data recording iVIMv^ftts) 
medium thus constructed into the computer, differential 
decoding of the interlacing scanned image or the like, which 
is adapted to differential coding with no degradation of 
coding efficiency resulting from coding, is implemented by 
software. 
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Figure 14 is a block diagram showing an image decoding 
apparatus according to a fourth embodiment of the present 
invention . 

as* 

Figure 15 is a block diagram for explaining image 
decoding apparatus according to a modification 1 of the 
fourth embodiment . 

Figure 16 is a block diagram showing an image decoding 
apparatus according to a modification 2 of the fourth 
embodiment . 

Figures 17(a), 17(b), and 17(c) are diagrams for 
explaining a data recording medium which contains a program 
for implementing image processing by the digital image 
padding method or in the image processing apparatus, in a 
computer system. 

Figure 18 is a diagram showing replacement of pixel 
values in a prior art digital image padding method. 

Figure 19 is a diagram showing averaging of pixel values 
in the prior art digital image padding method. 

Figure 20 is a diagram showing -resampling of pixel 
values in the prior art digital image padding method. 

Be e t Mode -f or C - triiyiny — Orrt — trtre invention 

Now, preferred embodiments of the present invention 
will be described with reference to figures 1-17. 
Embodiment l . 
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Figures 1-3 are diagrams for explaining a digital image 
padding method according to a first embodiment of the present 
invention. Figure 1 shows flow of processing of an image 
signal in the padding method. 

The digital image padding method of the first embodiment, 
as shown in figure 1, includes resampling an original image 
space formed by a sampled input image signal to form plural 
small image spaces (step S12) , and performing padding to each 
of the small image spaces by replacing sample values of 
insignificant pixels with padding pixel values obtained by 
a prescribed method. 

Figure 2 schematically shows resampling according to 
the digital image padding method. An original image space 
301 corresponding to the input image signal, comprises 8 x 
8 pixels, which are significant pixels having significant 
sample values and insignificant pixels having insignificant 
sample values. In figure 2, each pixel is represented by 
a square. The significant pixels are pixels (for example, 
pixels 303 and 304)of squares which are hatched, and the 
insignificant pixels are pixels ( for example pixel 302 ) which 
are not hatched. The numbers of respective squares indicate 
the corresponding sample values. 

Small image spaces 401-404 resulting from the 
resampling each comprises 4x4 pixels. In this embodiment, 
the small image space 401 comprises pixels, which are 
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Note that the digital image signal may correspond to 
the interlacing scanned image (interlaced image) or the 
non-interlacing scanned image (progressive image) . Where 
the digital image signal corresponds to the progressive image, 
for example, an image of an object of a horizontal stripe 
pattern corresponds to the image in the original image space 
301 in figure 2. On the other hand, where the digital image 
signal corresponds to the interlaced image, an image which 
moves greatly corresponds to the image of the original image 
space 301. In this interlaced image, as shown in figure 2, 
it sometimes looks like the object has a shape of two outlines. 

Sjs^iretes, the original image space 301 may be a region 
of the object (object region) itself comprising the object, 
or may be one of plural image spaces (blocks) each comprising 
a prescribed number of pixels (for example N x M samples), 
into which the object region is divided. In this embodiment, 
for convenience, assume that digital, image signals- of 
object regions are sequentially input frame by frame in step 
Sll . 

In step S12, the original image space 301 of the digital 
image signal is resampled to generate plural small image 
space 401-404. Specifically, every other pixels are 
extracted in the horizontal direction and pixels on every 
other pixel row are extracted in the vertical direction in 
the original image space 301, to generate the four small image 
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changed into a small image space 406 comprising pixels having 
sample values in figure 3(b). 

Comparing these small image spaces resulting from 
padding to the odd-numbered field image 381 and the 
even-numbered field image space 382 (see figure 20) obtained 
by the prior art method, sample values of pixels in the small 
image spaces 405 and 406 (see figure 3) are uniform, which 
shows that high-frequency components^ not introduced into 
the image signals of these small images. 

In final step S14, it is decided whether or not the 
resampled original image space corresponds to a final frame 
of the digital image signal. • When decided that the original 
image space corresponds to the final frame, process in steps 
S11-S14 are completed (Step S15) , whereas when decided that 
the original image space does not correspond to the final 
frame, the steps S11-S14 are performed to an original image 
space corresponding to a subsequent frame. 

Thus, in the first embodiment, every other pixels are 
extracted in the horizontal direction and pixels on every 
other pixel row are extracted in the vertical direction of 
the original image space 301 corresponding to the input image 
signal, and then the extracted plural pixels are rearranged 
to form 4 small image spaces 401-404, to which padding of 
pixel values are performed. Therefore, padding is 
performed to plural small image spaces where there is high 
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Subsequently, modification 1-3 of the first embodiment 
will be described. 

Modification 1 of First Embodiment 

Figure 4 is a diagram for explaining a digital image 
padding method according to a modification 1 of the first 
embodiment of the present invention and shows flow of image 
processing in the padding method. 

In this digital image padding method of the modification 
1 of the first embodiment /process for merging pixels in step 
S16 is added to the steps S11-S15 in the digital padding method 
of the first embodiment in figure 1. 

In step S16, all the pixels in the small image spaces 
405 and 406 which have been padded in step S13 are rearranged 
so that they have the same arrangement as the original image 
space corresponding to the input digital image signal, to 
form a merged image space 407 which is as large as the original 
image space 301 (figure 3(c)). 

Specifically, pixels in the small image space 405 
obtained by padding the small image space 401 are disposed 
where odd-numbered pixel columns and odd-numbered pixel rows 
intersect in the merged image space which is twice as large 
as the small image space, and pixels in the small image space 
405 obtained by padding the small image space 403 are disposed 
where even-numbered pixel columns an.d odd-numbered pixel 
rows intersect in the merged image space. Further, pixels 
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of the present invention, and shows flow of image processing 
in the padding method. 

The digital image padding method of the modification 
2 of the first embodiment, as shown in figure 5, includes 
step S52 in which the degree of correlation of pixel values 
between adjacent pixel columns is evaluated for an original 
image space formed by a resampled input digital image signal, 
step S53 in which it is decided whether or not the original 
image space is resampled to form plural small image spaces 
according to the evaluation result, step S54 in which the 
original image space is resampled and then the small image 
space is padded, and step S55 in which the original image 
space is directly padded. 

Subsequently, operation and effect will be described. 

In the padding method of the modification 2 of the first 
embodiment, in step S51, a digital image signal forming a 
prescribed original image space where pixels are arranged 
in^matrix is input. Subsequently in step S52, the degree 
of correlation of pixel values between adjacent pixel columns 
is computed for the original image space. More specifically, 
in this modification, differences of sample values of pixels 
between each pair of adjacent odd-numbered and even-numbered 
pixel columns in the original image space are computed, and 
absolute values of the differences are added for each pair 
of odd-numbered and even-numbered pixel columns, to 
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generate first evaluation values. 

In addition, differences (first differences) of sample 
values between each pair of adjacent odd-numbered pixel 
columns and differences (second differences) of sample 
values between each pair of adjacent even-numbered pixel 
columns are computed, and absolute values. of these first and 
second differences are added for each pair of even-numbered 
pixel columns and for each pair of odd-numbered pixel columns, 
to generate second evaluation values. 

In step S53, comparison is made between the first and 
second evaluation values. From this comparison result, 
when the second evaluation values are smaller than the first 
evaluation values, that is, correlation of pixel values 
between the odd-numbered and even-numbered pixel columns is 
lower than correlation of pixel values between odd-numbered 
pixel columns and between even-numbered pixel columns, in 
step S54, the original image space is resampled, and then 
padded. Specifically, pixels on every other pixel column 
are extracted in the original image space, to generate a first 
small image space comprising pixels forming odd-numbered 
pixel columns and a second small image space comprising 
pixels forming even-numbered pixel columns, and then padding 
of pixels is performed to each small image space. 

On the other hand, when the second evaluation values 
are not smaller than the first evaluation values from the 
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evaluation values, and the other functions may be employed. 
Moreover, the evaluation values may be computed only from 
significant samples. 

Modification 3 of First Embodiment 

Figure 6 is a diagram showing a digital image padding 
method according to a modification 3 of the first embodiment 
and shows flow of image processing in the padding method. 

The digital image padding method of the modification 
3 of the first embodiment, as shows in figure 6, includes 
step S62 in which the degree of correlation of pixel values 
between adjacent scanning lines (pixel rows) is evaluated 
for an original image space formed by a sampled input digital 
image signal, step S63 in which the degree of correlation 
of pixel values in fields of the original image space is 
decided, step S64 in which the original image space is 
resampled to form a small image space corresponding to an 
odd-numbered field and a small image space corresponding to 
an even-numbered field, and padding is performed for each 
small image space, i.e., field by field, and a step S65 in 
which the original image space is padded frame by frame. 

Subsequently, operation and effects will be described. 
In the padding method of the modification 3 of the first 
embodiment, in step S61, a digital image signal forming a 
prescribed original image space in which pixels are arranged 
ir/matrix is input. In this modification 3, suppose that 
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the digital image signal corresponds to the interlaced image, 
and forms the image space 301 shown in figure 2. 

Subsequently in step S62, correlation of pixel values 
between scanning lines in the original image space is 
computed. Specifically, by the method described by means 
of figure 5, first evaluation values indicating correlation 
of pixel values between odd-numbered and even-numbered lines 
in the original image space are generated, and second 
evaluation values indicating correlation of pixel values 
between odd-numbered lines and between even-numbered lines 
in the original image space are generated. 

Then in step S63, comparison is made between the first 
and second evaluation values, to decide whether or not 
correlation of pixel values in fields is higher than 
correlation of pixel values in a frame. 

When decided that the second evaluation values are 
smaller-than the first values, that is, correlation of pixel 
values in the fields is higher than correlation of pixel 
values in the frame, in step S64, the original image space 
formed by the digital image signal is padded field by field. 
In other words, a small image space composed of odd-numbered 
lines and a small image space composed of even-numbered lines 
are generated, and then padding is performed for each small 
image space (field) . On the other hand, when decided that 
the second evaluation values are not smaller than the first 
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values, in step S65, the original image space is padded frame 
by frame, that is, with the odd-numbered lines and the 
even-numbered lines merged. 

Figure 7 is a diagram for explaining field by field 
padding of the original image space 301 in figure 2. Figure 
7 (a) shows a small image space 701 corresponding to the 
odd-numbered field in the original image space 301, and a 
small ij£&lr<fe space 703 obtained by padding the small image 
space 701. Figure 7(b) shows a small image space 702 
corresponding to an even-numbered field in the original image 
space 301, and shows a small image space 704 obtained by 
padding the small image space 702. 

As can be seen from figure 7, the small image spaces 
701 and 702 can be padded without introducing high-frequency 
components into them. 

In this modification 3 of the fixst embodiment, it is 
decided -whether the original image space corresponding to 
the input image digital image signal is padded field by field 
or frame by frame, and padding of pixel values is performed 
to the original image space in one of these units . Therefore, 
it is possible to perform padding to the interlaced image 
without degrading efficiency in coding of the image signal 
of to the interlaced image. 
Embodiment 2 . 

Figure 8 is a diagram for explaining a digital image 
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padding method according to a second embodiment of the 
present invention, and shows an original image space formed 
by a digital image signal to be padded, and a padded image 
space in which the original image space has been padded. 

In the digital image padding method of the second 
embodiment, the step S12 for resampling and step S13 for 
padding by the digital image padding method of the first 
embodiment, are replaced by a step in which an original image 
space 801 corresponding to a digital image signal is directly 
padded, that is, sample values of insignificant pixels are 
replaced with padding values obtained from sample values of 
significant pixels according to a prescribed rule. In this 
embodiment, used as the significant pixels used in generating 
the padding values are, pixels which are adjacent to 
insignificant pixels in the horizontal direction and pixels 
which are adjacent thereto one pixel spaced apart therefrom 
in the vertical direction of the original image space. 

Specifically, the digital image padding of the second 
embodiment includes a first replacement step S21 (see figure 
8(a)) for sequentially replacing sample values of 
insignificant pixels in the horizontal direction and a second 
replacement step S22 (see figure 8(b)) for replacing sample 
values of insignificant pixels in the vertical direction. 
In the second replacement step S22, sample values of 
insignificant pixels in an image space 801a obtained through 
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the first replacement step ^ti , are "replaced with sample 
values of pixels on a pixel row adjacent to a pixel row 
comprising the insignificant pixels, one pixel row spaced 
apart therefrom. 

Subsequently, operation and effects will be described. 
Since there is low correlation of pixel values between 
an odd-numbered line (pixel row) and an even-numbered line 
(pixel row) in the original image space 801, effectiveness 
is provided by the method of dividing the original image space 
801 into the small image space composed of odd-numbered lines 
and the small image space composed of even-numbered lines, 
and padding respective small image spaces, as already 
described in the first embodiment. On the other hand, for 
an image signal forming an original image space with high 
correlation between odd-numbered lines and between even- 
numbered lines, there is another effective method, in which 
sample values to be padded, i.e., insignificant pixels in 
the original image space, are replaced with sample values 
of significant pixels located one pixel spaced apart 
therefrom in the vertical direction (one pixel is located 
adjacently to the other pixel with one pixel interposed 
between them in the vertical direction) . 

So, in the digital padding method of the second 
embodiment, as shown in figure 8(a), sample values of 
insignificant pixels are replaced with sample values of 
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embodiment . 
Embodiment 3 . 

Figure 9 is a block diagram showing the image coding 
apparatus as an image processing apparatus according to the 
third embodiment of the present invention, wherein figure 
9(a) shows an entire construction of the image coding 
apparatus and figure 9 (b) shows a padding section of the image 
coding apparatus. 

An image coding apparatus 100a of the third embodiment 
includes a blocking unit 101 for dividing an input digital 
image signal Sg supplied to an input terminal 101a into 
signals corresponding to blocks each comprising 16 x 16 
pixels as coding units and outputting the resulting image 
signal (hereinafter referred to as a blocked image signal) 
Bg, an interlace detecting unit 104 for distinguishing 
between the interlaced image and the progressive image on 
the basis of an output Bg of the blocking unit 101, for an 
image corresponding to the digital image Sg, and outputting 
a sample identifier Di according to the result, and a 
predicting section 120a for producing a prediction image Prg 
of a target block to-be-coded. 

The image coding apparatus 100a further includes a 
subtracter 102 for outputting difference between the blocked 
image signal Bg of the target block and the prediction signal 
Prq as a difference signal (prediction error signal) Dg, data 

A 
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compressing m&ecf\s 110 for performing information compression 
to the output Dg of the subtracter 102 according to the sample 
identifier Di and outputting a compressed difference signal 
Qg, and a variable length encoder (VLC) 103 for performing 
variable length coding to the output Qg of the data 
compressing rj^ans 110 and outputting a coded image signal 

Eg- 

The data compressing m^rrfs 110 includes a pixel 
rearrangement unit 113 for rearranging pixels of a difference 
image signal formed by the difference signal Dg according 
to the sample identifier Di, a DCT unit 111 for performing 
discrete cosine transform to an output PDg of the pixel 
rearrangement unit 113, and a quantizer 112 for quantizing 
an output Tg of the DCT unit 111 and outputting quant izat ion 
coefficients as the compressed difference signal Qg . The 
discrete cosine transform is performed for each of 4 blocks 
each comprising 8x8 pixels into which the block comprising 
16 x 16 pixels is divided. Where the rearrangement has been 
performed, a block 250c is divided into small blocks 251-254 
as shown in figure 10(c), whereas where the rearrangement 
has not been performed, a block 250b is divided into small 
blocks 255-258 as shown in figure 10(b). 

The pixel rearrangement unit 113 is used for rearranging 
pixels in the image space 250 corresponding to the difference 
signal Dg to generate a rearranged image space 250a according 
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to the sample identifier Di as shown in figure 10(a). In 
this rearranged image space 250a, an upper region 250al 
comprising 16 x 8 pixels comprises pixels on odd-numbered 
pixel rows and a lower region 250a2 comprising 16x8 pixels 
comprises pixels on even-numbered pikel rows in the image 
space 2 50. 

The predicting section 120a includes data expanding 
rrus-ans 130 for performing expansion to the output Qg of the 
data compressing r^ea-ris 110 and outputting an expanded 
difference signal Itg, and an adder 123 for adding the 
expanded difference signal Itg to the prediction image signal 
Prg and outputting a restored image signal Lrg . The data 
expanding rna-ams 130 includes an inverse quantizer 121 for 
inversely quantizing the output Qg of the data compressing 
jru&Stis 110, and an IDCT unit 122 for performing inverse DCT 
to an output IQg of the inverse quantizer 121 by transforming 
data in- a frequency domain into data in a spatial domain, 
and outputting the expanded difference signal Itg. 

The predicting section 120a further includes padding 
mg&ns 150 for performing padding to the restored image signal 
Lrg from the adder 123 according to the sample identifier 
Di, and prediction signal producing m^ms 140 for producing 
the prediction image signal Prg from a padded restored image 

signal Pag - efcc/c^ 

The prediction signal producing p^ans 140 includes a 
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frame memory 124 for storing the output Pag of the padding 
mg-a-rrs 150, a motion detecting unit (ME) 125 for computing 
a motion vector MV of the target block based on the image 
data Mg stored in the frame memory 124, the blocked image 
signal Bg, and the sample identifier Di and outputting the 
MV, and a motion compensating unit (MC) 126 for generating 
an address Addl of the frame memory 124 according to the motion 
vector MV from the motion detecting unit 125 and outputting 
an image signal in a memory area of the address Addl as the 
prediction signal Prg. The motion detecting unit 125 is used 
for generating a motion vector of a frame and a motion vector 
of a field according to the sample identifier Di . The motion 
vector MV and the sample identifier Di are variable length 
coded by the variable length encoder 103 together with the 
compressed difference signal Qg . 

As shown in figure 9 (b) , the padding jris-ems 150 includes 
a frame padding unit 128a for padding the output Lrg of the 
adder 123 frame by frame, a field padding unit 128b for padding 
the restored image signal Lrg from the adder 123 field by 
field, a primary switch 127a provided at a primary stage of 
these padding units, for supplying one of the padding units 
with the restored image signal Lrg from the adder 123, a 
secondary switch 127b provided at a next stage of these 
padding units, for selecting one of outputs Pal and Pa2 of 
respective padding units according to the sample identifier 
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Di, and a pixel rearrangement unit 129 for receiving a padded 
restored signal as a selected signal Spa from the secondary 
switch 127b and rearranging pixels in an image space formed 
by the padded restored image signal. 

As shown in figure 10(b), the frame padding unit 128a 
is used for performing padding the restored image signal Lrg 
of 4 small blocks (8x8 pixels) 255-258 in an image space 
250b (16 x 16 pixels) corresponding to the restored image 
signal at a time. As shown in figure 10 (c) , the field padding 
unit 128b is used for performing padding to the restored image 
signal Lrg of small blocks 251 and 252 composed of first field 
lines in an upper region 250cl of an image space 250c of the 
restored image signal at a time, and padding the signal Lrg 
of small blocks 253 and 254 composed of second field lines 
in a lower region 250c2 of the image space 250c at a time. 
The pixel rearrangement unit 129 is used for rearranging 
pixels in an image space 250d corresponding to a padded 
restored image signal from the field padding unit 128b and 
outputting the image signal Pag corresponding to a merged 
restored image space 250e where 8 pixel rows in the upper 
region 250dl and 8 pixel rows in the lower region 250d2 of 
the image space 250d are alternately arranged. 

Next, data structure of a coded image signal output from 
the image coding apparatus 100a will now be described. 
Figure 11 schematically shows a data structure of the coded 
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image signal (bit stream) . 

In this bit stream 200, a bit string F corresponding 
to a frame image comprises a synchronous image signal 203 
indicative of a starting point of a code string, a 
quantization step value 204 indicative of a quantization 
scale (quantization step) for use by quantization in coding 
process, and data D(l) , D(2) , (D3) , . . . , D(i) , corresponding 
to blocks (macroblocks) MB ( 1 ) , MB (2), HBO),..., MB ( i ) each 
comprising 16 x 16 pixels. Reference numerals 205, 208, and 
211 designate sample identifiers each indicates whether or 
not correlation between pixels in fields is higher than 
correlation between pixel values in a frame. Reference 
numerals 206, 209, and 212 designate variable length coded 
motion vectors. Reference numerals '207, 210, and 213 
designate DCT coefficients of subblocks each comprising 8 
x 8 pixels of each block. The data D(l), data D (2), and 
D(3) corresponding to respect ive macro blocks comprises , the 
sample identifier 205, the motion vector 206, and the DCT 
coefficient 207 , the sample identifier 208 , the motion vector 
209, and the DCT coefficient 210, and the sample identifier 
211, the motion vector 212, and the DCT coefficient 213, 
respectively. In figure 11, reference numeral 214 
designates a synchronous image signal of a frame subsequent 
to the frame F. 

In this embodiment, the synchronous image signals 203 
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When the difference signal Dg is supplied to the data 
compressing me*tTs 110, the pixel rearrangement unit 113 
subjects the difference signal Dg to rearrangement according 
to the sample identifier Di . Specifically, where the 
digital image signal corresponds to the interlaced image, 
the difference, signal Dg is. subjected to rearrangement and 
input to the DCT unit 111, whereas where the digital image 
signal corresponds to the progressive image, the difference 
signal Dg is directly input to the DCT unit 111 without 
rearrangement. The DCT unit 111 performs DCT to the output 
PDg of the pixel rearrangement unit 113. More specifically, 
for the block corresponding to the interlaced image, the 
block is divided into subblock as shown in figure 10 (c) , and 
for the block corresponding to the progressive image, the 
block is divided into subblocks as shown in figure 10(b), 
and then the output PDg is subjected to DCT for each subblock. 
The quantizer 112 quantizes an output Tg of the DCT unit 111 
by a quantization step and outputs quantization coefficients 
Qg of the difference signal to the variable length encoder 
103 and the information expanding section 130. 

The information expanding section 130 expands and 
converts the compressed difference signal Qg of the target 
block into the expanded difference signal ITg. More 
specifically, in this embodiment, the inverse quantizer 121 
inversely quantizes the compressed difference signal Qg , and 
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further, the inverse discrete cosine transform (IDCT) unit 
122 subjects the output IQg of the quantizer 121 to IDCT by 
transforming a signal in a frequency domain into a signal 
into a signal in a spatial domain. 

The adder 123 adds the expanded difference signal ITg 
to the prediction image signal Prg and outputs the restored 
image signal Lrg to the padding means 150. 

In the padding qxe^ms 150, the restored image signal Lrg 
is supplied to one of the frame padding unit 128a and the 
field padding unit 128b through the switch 127a. At this 
time, switching of the switch 127a is controlled according 
to the sample identifier Di . Where a value of the sample 
identifier Di is 1, that is, there is high correlation between 
pixels in the fields, the restored image signal Lrg is 
supplied to the field padding unit 128b through the switch 
127a and is subjected to intra-field padding therein. 
Specifically, as shown in figure 10(c) , padding is performed 
to the restored image signals of small blocks 251 and 252 
composed of lines of the first field of the restored image 
space 250c at a time, and the restored image signals of small 
blocks 253 and 254 composed of lines of the second field of 
the restored image space 250c at a time. 

On the other hand, where the value of the sample 
identifier Di is 0, that is, where there is high correlation 
between pixel values in the frame, the restored image signal 
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progressive image. Therefore, it is possible to perform 
differential coding to the interlaced image with high 
efficiency . 

Subsequently, modifications of the third embodiment 
will be described. 

Modification 1 of Third Embodiment 

Figure 12 is a diagram showing an image coding apparatus 
according to a modification 1 of the third embodiment. 

Iri an image coding apparatus 100b of the modification 
1 of the third embodiment, only a predicting section 120b 
differs from that of the image coding -apparatus 100a of the 
third embodiment, and in the predicting section 120b of the 



predicting section 120a of the first embodiment is replaced 
with a merging unit 151 which performs merging wherein pixels 
in an image space formed by the restored image signal Lrg 



from the adder 123 are rearranged, and^padding joae^ns 150b 
which performs padding to an output of the merging unit 151. 
/ The merging unit 151 is, as shown in figure 10 (d) , used 

for rearranging pixels by alternately arranging pixels rows 
in the upper region 250dl and pixel rows in the lower region 



250d2 of the image space 250d. The padding pw=?irfis 150b 
includes the primary and secondary switches 127a and 127b, 
the frame padding unit 128a, and the field padding unit 128b 
of the padding m&^s^50 of the third embodiment, the field 



image coding apparatus 10 0b , 
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padding unit 128b performing padding by the digital image 
padding method of the second embodiment- The other 
components are identical to those of the third embodiment. 

In the modification 1 of the third embodiment thus 
constructed, the same effects as provided by the third 
embodiment are obtained. 
Modification 2 of Third Embodiment 

$ Figure 13 is a diagram showing an image coding apparatus 

according to a modification of the third embodiment of the 
present invention, wherein figure 13(a) shows an entire 
construction of the image coding apparatus and figure 13(b) 
shows^padding nutans 150c of the data compressing rosins 110c 
of the image coding apparatus. 
f<< In an image coding apparatus 10 0'c of the modification 

2 of the third embodiment, only data compressing j^e5ns 100c 
differs from that of the image coding apparatus 100a of the 
third embodiment, and the data compress ing jjiea-rrs 110c of the 
image coding apparatus 100c is provided with padding m^aTis 
150c which performs padding to an output PDg of the pixel 
rearrangement unit 113 between the pixel rearrangement unit 
113 and the DCT unit 111 at the next stage of the data 
compressing rj^arfas 110 of the third embodiment . 
^ The padding m^^^Oc includes a frame padding unit 
158a for performing padding to the output PDg of the 
rearrangement unit 113 frame by frame, a field padding unit 
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158b for performing padding to the output PDg of the 
rearrangement unit 113 field by field, a primary switch 157a 
provided at a previous stage of these padding units, for 
supplying one of these padding units with the output PDg 
according to the sample identifier Di, and a secondary switch 
157b provided at a next stage of these padding units, for 
selecting one of outputs PDgl and PDg2 of the padding units 
according to the sample identifier Di . Constructions of the 
frame padding unit 158a and the field padding unit 158b are 
identical to those of the frame padding unit 128a and the 
field padding unit 128b of the third embodiment. 

In this modification 2 of the third embodiment, since 
padded difference signal is supplied to the DCT unit 111, 
efficiency in DCD process is increased, in addition to the 
effects provided by the third embodiment. 
Embodiment 4 . 

Figure 14 is a block diagram showing an image decoding 
apparatus according to a fourth embodiment of the present 
invention . 

An image decoding apparatus 900a of the fourth 
embodiment is used to receive a bit stream 200 of the data 
structure in figure 11 as a coded image signal Eg in which 
an image signal has been compressively coded and perform 
decoding to the coded image signal Eg. 

Specifically, the image decoding apparatus 900a 
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includes a data analyzing unit 902 for receiving a coded 
difference signal Eg of a target block to be decoded (target 
block) , and performing analysis and variable length decoding 
to the coded difference signal Eg/data expanding japans 903 
for expanding an output AEg of the data analyzing unit 902 
and outputting an expanded image signal DEg, an adder 906 
for adding the output DEg to a prediction image signal Pg2 
of the target block, an inverse blocking unit 907 for 
integrating reproduced image signals. of respective blocks 
as an output PEg of the adder 906 and outputting an image 
output signal REg of a scanning line structure, and a 
predicting section 910a for producing the prediction image 
signal Pg2 . 

'4- The data expanding jpxe^ns 903 comprises an inverse 

quantizer 904 for inversely quantizing the output AEg of the 
data analyzing unit 902, and an IDCT unit 905 for performing 
IDCT (inverse discrete cosine transform) as one type of 
inverse frequency transform, to the output DEgl of the 
inverse quantizer 904, and outputting the expanded 
difference image signal DEg. 

^ I The predicting section 910a includes padding m>ans 920a 

for performing padding to the output PEg of the adder 906, 
a frame memory 909 which contains an output Pa of the padding 
x^sTas 920a as a reproduced image signal, and a motion 
compensating unit 911 for generating an address Add2 of the 
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frame memory 909 according to an output Rg2 of the frame memory 
909 and a motion vector MV of the block which has been decoded 
by the data analyzing unit 902, and reading a prediction image 
signal Pg2 of the target block from the frame memory 909 which 
contains recorded data Rg2 using the address Add2 . 

The padding nue-a~ns 920a includes a frame padding unit 
921a for performing padding to the output PEg of the adder 
906 frame by frame, a field padding unit 921b for performing 
padding to the output PEg field by field, a primary switch 
922 provided at a previous stage of these padding units, for 
supplying one of them with the output PEg according to the 
sample identifier Di from the data analyzing unit 902, a 
secondary switch 923 provided at a next stage thereof, for 
selecting one of outputs Pal and Pa2 of respective padding 
units according to the sample identifier, and a pixel 
rearrangement unit 924 for rearranging pixels of the padded 
image signal as a selected signal Pas from the secondary 
switch 923. 

Besides, reference numerals 901a and 901b designate 
input and output terminals of the image decoding apparatus 
90 0a, respectively. 

Subsequently, operation will be described. 

When" the compressively coded image signal Eg, for 
example, the bit stream 200 in figure 11 is input to the input 
terminal 901a, the data analyzing unit 902 performs variable 
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length decoding to the coded image signal by analysis. The 
resulting decoded data is output t.o the data expanding jjue-afis 
903 as a compressed difference signal AEg in which a 
quantization step or a DCT coefficient has been compressed. 
The data analyzing unit 902 also outputs the motion vector 
MV and the sample identifier Di to the predicting section 
910a . 

The data expanding gj^stis 903 expands the compressed 
difference signal to be restored to an expanded difference 
signal of the target block. Specifically, the inverse 
quantizer 904 inversely quantizes the compressed difference 
signal, and then the IDCT unit 905 performs inverse frequency 
transform to the output DEgl of the quantizer 904, by 
transforming a signal in a frequency domain into a signal 
in a spatial domain. 

The motion compensating unit 911 generates the address 
Add2 for accessing the frame memory 909 according to motion 
vector MV from the data analyzing unit 902 and extracts the 
prediction image signal Pg2 of the target block from the 
reproduced image signal stored in the frame memory 909. The 
adder 906 adds the output DEg of the data expanding ros-arns 
to the prediction image signal Pg2 and outputs the reproduced 
image signal PEg of the target block. 

The reproduced image signal is supplied to the inverse 
blocking unit 907 and the predicting section 910a. 
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those of the image decoding apparatus 900a of the fourth 
embodiment in figure 14. 

In the image decoding apparatus 900b, the predicting 
section 910a of the image decoding apparatus 900a of the 
fourth embodiment is replaced by a predicting section 910b 
which differs in construction from the same. 

In the predicting section 910b, there is provided 
''padding m^ans 920b between a motion compensating unit 911 
and an adder 906, for performing padding to a motion 
compensated prediction image signal Pg2 and supplying the 
resulting signal to the adder 906. The padding jjieens 920b 
includes a frame padding unit 921a, a field padding unit 921b. 
a primary switch 922 for supplying one of these padding units 
with the prediction image signal Pg2 from the motion 
compensating unit 911 according to the sample identifier Di, 
and a secondary switch 923 for selecting one of outputs of 
the padd-ing units according to the sample identifier Di and 
outputting a selected signal to the adder 906. 

Subsequently, operation and effects will.be described. 

Motion compensation is performed to the interlaced 
image frame by frame or field by field. The frame by frame 
motion compensation is performed by obtaining a prediction 
image signal from a frame, while the field by field motion 
compensation is performed by obtaining prediction image 
signals from the corresponding first and second fields 
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separately. 

The image decoding apparatus 900b according to the 
modification 1 of the fourth embodiment provides 
effectiveness in the field by field motion compensation . In 
the image decoding apparatus 900b, switching of the primary 
switch 922 is controlled according to the sample identifier 
Di from the data analyzing unit 902, and the output of the 
motion compensating unit 911 is input to one of the frame 
padding unit 921a and the field padding unit 921b. The 
prediction image signal is padded by one of these padding 
units and supplied to the adder 906. 

In this modification 1 of the fourth embodiment, since 
padding is performed to the prediction image signal fetched 
from the frame memory 909, amount of signals to-be-padded 
in the predicting section can be reduced, as compared to a 
case where padding is performed to all the reproduced image 
signals -stored in the frame memory 909. 
Modification 2 of Fourth Embodiment 

Figure 16 is a block diagram showing an image decoding 
apparatus according to modification 2 of the fourth 
embodiment. An image decoding apparatus 900c according to 
the modification 2 of the fourth embodiment has constructions 
of the image decoding apparatus 900a of the fourth embodiment 
and the image decoding apparatus 900b of its modification 
1 in combination. In other words, in addition to the 



77 



predicting section 910a comprising the padding jjie-atis 920a 
of the image decoding apparatus 900a of the fourth embodiment, 
the image decoding apparatus 900c includes auxiliary padding 
roS^trs (second padding joae-aTis ) 920c2 provided at a next stage 
of the motion compensating unit 911, the output of which is 
output to the adder 906. Here it is assumed that 
constructions of the first and second padding rjpe^ns 920cl 
and 920c2 are identical to those of the padding rrie^ns 920a 
of the fourth embodiment and the padding rae^ns 920b of the 
modification 1 of the fourth embodiment, respectively. 

In the modification 2 of the fourth embodiment thus 
constructed, the first and second padding jnea:ns 920cl and 
920c can share padding, whereby padding can be performed in 
the predicting section with higher efficiency. 

Furthermore, a program for implementing the digital 
image padding method or the image processing apparatus shown 
in each of the embodiments is recorded in a data recording 
medium such as a floppy disc, whereby processing is performed 
with ease in an independent computer system. 

Figure 17 is a diagram showing a case where image 
processing of the digital padding method, or image coding 
or image decoding of the image processing apparatus in the 
above embodiments is performed in a computer system by the 
use of a floppy disc which contains a program of these image 
processing. 
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Figure 17 (b) shows a front appearance and a cross- 
section of a floppy disc FD, and a floppy disc body as a 
recording medium. Figure 17(a) shows a physical format of 
the floppy disc body D. The floppy disc body D is stored 
in a case F, and in a surface of the disc body D, plural tracks 
Trs are formed from outer to inner radius thereof, each track 
being divided into 16 sectors Se in angle direction. 
Therefore, in the floppy disc body D which contains the 
program, data of the program is recorded in an allocated 
region of the floppy disc FD. 

Figure 17(c) shows a construction of recording and 
reproducing the program in/from the floppy disc FD. In case 
of recording the program in the floppy disc FD, the data of 
the program is written thereto through the floppy disc drive 
FDD from the computer system Cs . In another case of 
constructing the image decoding method in the computer system 
Cs by the program in the floppy disc FD, the program is read 
from the floppy disc FD by the use of the floppy disc drive 
FDD and transferred to the computer system Cs . 

While description has been given of image processing 
in a computer system by the use of the floppy disc as the 
data recording medium, this image processing can be also 
performed by the use of an optical disc. Besides, the 
recording medium is not limited to this, and an IC card, a 
ROM cassette, and so forth, may be used so long as they can 
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record a program therein. 

Industrial h™* "* 1 1 i -*~ y 

As described above, the digital image padding method, 
the image processing apparatus, and the data recording medium 
of the present invention, can increase coding efficiency in 
compression of an image signal, they are very useful in 
implementing coding or image decoding in a system which 
transmits/stores the image signal, and simultaneously they 
are suitable for compression and expansion of a moving 
picture according to a standard such as MPEG4 . 
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ABSTRACT 



In a digital image padding method according to the 
present invention, as shown in figure 2, every other pixels 



digital image signal are extracted in the horizontal 
direction, and every other pixels are extracted in the 
vertical direction, the extracted plural pixels are 



pixels values in these small image spaces are padded. 

In the digital image padding method thus constructed, 
padding of pixel values is performed to plural small image 
spaces where there is high correlation of pixel values 
between adjacent scanning lines rather than to an original 
image space where there is low correlation of pixel values 
between -adjacent scanning lines. That is, in the padded 
small image spaces, there is high correlation of the pixel 
values. As a result, it is possible to avoid introduction 
of high-frequency components into the digital image signal 
of the interlaced image, and hence, differential coding or 
differential decoding can be performed to the interlaced 
image or the like with high coding efficiency while 
suppressing degradation of coding efficiency resulting from 
padding. 



in an original image 




corresponding to an input 



rearranged to form four small image 




and 



Version with Markings to Show Changes Made 

1 . (Amended) A digital image padding method in which padding of pixel values is performed 
to a digital image signal forming an original image space including [comprising] an image having an 
arbitrary shape and containing [comprising] significant pixels and insignificant pixels, said method 
comprising: 

a pixel rearrangement [step] in which plural pixels in the original image space are grouped 
[according to a prescribed rule] to form plural small image spaces each comprising pixels of the same 
group so that a pixel value correlation within the small image spaces becomes higher than a pixel 
value correlation within the original image space ; and 

apixel padding [step] in which values of insignificant pixels in each of the small image [space] 
spaces obtained by said pixel rearrangement are replaced with padding pixel values generated on the 
basis of the significant pixel values in each of the small image spaces [by a prescribed method]. 

2 . (Amended) The digital image padding method of claim 1 A wherein in said pixel rearrangement 
[step], sampling is performed by extracting pixels every (N+l)-th (N isa [:] positive integer) pixel 
in a prescribed direction of the original image space (N+ 1 ) times by using [the] first to [the] (N+ 1 ) -th 
pixels as starting sample pixels[,] to form (N+l) pieces of small image spaces each comprising a 
prescribed number of pixels obtained by one padding. 

3 . (Amended) The digital image padding method of claim 2^ wherein operation values obtained 
by performing operation on values of significant pixels in an n-th (n=l, 2, . . ., N+l) small image 
space are used as the padding pixel values for replacing the values of insignificant pixels in the n-th 
small image space. 

6. (Amended) The digital image padding method of claim 1 ^ wherein in said pixel rearrangement 
[step], sampling is continuously performed twice using [the] first and second pixel rows as starting 
sample pixel rows [,] by extracting pixels on every other pixel row in a [the] vertical direction of the 
original image space [J to form first and second small image spaces each comprising a prescribed 
number of pixels obtained by first and second samplings, respectively. 



7. (Amended) A digital image padding method in which padding of pixel values is performed 
to an interlaced [a] digital image signal forming an original image space including [comprising] an 
image having an arbitrary shape and containing [comprising] significant pixels and insignificant pixels, 
said method comprising [the steps of]: 

dividing the original image space into a first small image space comprising pixels on odd- 
numbered pixel rows in the original image space and a second small image space comprising pixels 
on even-numbered pixel rows in the original image space; and 

generating first padding pixel values from values of significant pixels in the first small image 
space and replacing values of insignificant pixels in the first small image space with the first padding 
pixel values, and generating second padding pixel values from values of significant pixels in the 
second small image space and replacing values of insignificant pixels in the second small image space 
with the second padding pixel values. 



